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ABSTRACT 

Although a large number of adults in the United 
States are assumed to need literacy training, little agreement exists 
on that aunber. Adults who attend literacy instruction are assumed to 
acquire the skills, but little is known about program effectiveness. 
Another assumption is that adults should define their goals, but this 
does not ensure that they strive for the levels of literacy they 
need. The primary reasons for literacy testing, from which data are 
regularly accessed by polic3nnakers , are as follows: national and 
state population assessments; individual skill assessments; placement 
procedures; and program evaluation. Tests of functional abilities 
favored for population surveys do not assess writing or mathematical 
ability and are incompatible with national surveys of elementary and 
secondary literacy abilities. Basic skills tests used for individual 
skill assessments report scores in grade-level equivalents, which are 
problematic for adults. Current placement testing methods are 
lengthy, and entering students' poor test-taking skills may lead to 
low reliability. Scores from program evaluations have relatively 
limited use for pol icjnnakers for a number of reasons: pretest scores 
tend to be artificially low; program evaluations often ignore writing 
and mathematics; and student goals do not always coincide with 
national standards. Grade-level reporting for test scores needs to be 
reconsidered. The materials and tasks approach would be easier to 
relate to home, work, and civic functioning; the cognitive processing 
approach would be easier to relate to elementary and secondary 
assessment and instruction. (Contains 16 references.) (YLB) 
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OVERVIEW 



The Departments ofEducation, Labor, andHeallh and 
Human Services now administer a variety of social pro- 
grams that incorporate literacy training. Adult Basic 
Education (ABE), English as a Second Language (ESL), 
JOBS, workplace literacy, family literacy, library literacy, 
and correctional institution educadon are the primary 
examples of these programs, but others exist that either 
provide literacy instruction directly or that incorporate 
literacy as a component of another social service. Central 
to the proper administration of these programs is <\ppropri- 
ate information on the needs and characteristics of the 
clients for these services, the quality of services actually 
delivered, and the impactof theservices upon the individu- 
als who receive them. Literacy tests are one of the most 
widely used (and controversial) means through which 
information on all three of these topics is obtained. Al- 
though tests are not and should not be the only means 
through which information is gathered, they are by tradi- 
tion the primary procedure through which comparative 
analyses have been performed. From national surveys of 
liteiticy, suchas theNational AdultLiteracy Survey (NALS) 
now underway, to locally developed screening and place- 
ment instruments, a melange of literacy tests now exists, 
providing data with widely varying degrees of reliability, 
validity, and comparability. 

This brief synthesis is an initialattempt to delineate the 
central issues in literacy testing and to define options that 
might provide an improved data base for social policy 
development The plan of this p^r is to attend first to 
basic issues in literacy testing: the perceived needs for 
literacy funding and the potential outcomes of literacy 
programs. Then various types of testing needs are dis- 
cussed, along with options for satisfying these need*^. 
Finally, the measures or scales used forreporting literacy 
levels are examined. 
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BASIC ISSUES 



Several key assumptions underlie current literacy policies, some of which have not been 
subjected to sufficient scrutiny. Although some of these assumptions have been studied and 
others are the subjects of ongomg studies, many questions remain. The rrst assumption is that 
a large number of adults in America need and would pcoGx from literacy training. Although few 
disagree with this assumption in its g^eral fonn, little agreement exists on the number of adults 
who need assistance or on their physical and mental characteristics. One of the barriers to 
resolving this debate is the lack of differentiation for literacy needs. On an absolute level, almost 
everyone could benefit from further literacy instruction, even lawyers and doctors. For policy- 
making, however, tiie most important needs are those that will allow people to be self-sufficient 
economically, to manage their homes, and to be functioning members of their communities. 

However, even within this domain, needs differ widely. Hie ESL needs of professional 
iinnugrantsfromEastemEuropearevastiydifferentfix)mthebasicUteracyneedsofsonienu 
school dropouts, and these differ from die literacy needs of high school graduates who want entry 
to advanccii technical skill tnuning. Thetypesof instruction that each of tiiesepopulationsneed 
to qualify tiiem for die same employment opportunities and die same levels of home and civic 
functioning have different resource and administrative demands. Effective policy requires more 
than a combined count of all who might profit from further literacy instruction. 

A second assumption is that if adults could only be persuaded to attend adult basic education 
programs, or odier types of literacy instruction, they would acquire die skills diey are presumed 
to be lacking. This assumption is relatively untested. We know littie about die effectiveness of 
different literacy programs and have hmited experience in measuring literacy program perfor- 
mance (Daikenwald, 1986). Almostall program outputmeasurcsare of acad'^corfunctional 
skills, yet most researchers agree tiiat attitudes andbeliefis also deserve atti4ition. Even the roles 
of some types of programs are in doubt Library tutoring programs, for example, usually provide 
services to adults at die lowest ability levels, using relatively untrained volunteers for tutors. Do 
we expect ^preciable literacy skill gains from such programs or do we expect more confidence 
building and survival skill acquisition? To what degree should diese programs prepare and 
encourage adults to attend ABE classes? 

The third and most important assumption is tiiat adults who acquire whatever is being taught 
in literacy programs willbebetter equipped fcffhigh-skilled jobs in dielabarmaiket, will function 
better as parents and home j^anagers, and will participate more fiiUy in dvic and community 
affairs. Many literacy programs operate on an c^n entry, open exit basic, wherein studaits 
define dieir own goals. Aldiough diis po' fcy may be successful in encouraging adults to attend 
programs diey might not be comfortable in under diflferent operating procedures, it does not 
ensure diatadults will be striving for die levels of literacy diat they need and could attain. Adults 
widi low literacy levels tend to lack confidence in dieir own abilities to learn. Widiout assistance 
in defining what dieir needs might be, diey niay aim f(x far less tiian what is necessary for their 
own success 

More serious is die lack of basic understanding of die literacy demands of home and civic 
functioning. We assume, for example, diat one goal of literacy instruction is to improve 
parenting, particularly for fostering die literacy devcl(^ent of children, yet wchave only crude 
estimates of the amounts and types of reading, writing, and mathematics that are required to 
succeed at this task. Like most of the odier goals of literacy instruction, parenting is dependent 
upon mudi more than literacy. In some cases, liteiacy is only a proxy for some complex of skills 
and may not be necessary, given compensating conditions. For example, many inunigrant and 
refugee parents in America who are not literate in English are neverdieless quite effective in 
fostering dieir children's education (Caplan, Choy, & Whitmore, 1992). 
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THE ROLE OF TESTING 



TTie primary reasons for literacy testing, frcMn which data are regularly accessed by 
policymakers, are: ( 1) national and state population assessments (e.g., NALS), (2) individual 
skill assessments, (3) placement procedures, some of which involve testing, and (4) program 
evaluation. • 

National and state surveys of literacy involve two types of data collection: background 
information, generally gathered orally, on educational, economic, health, language, and civic 
acti\nities and experiences; and skill abilities as measured by some nationally standardized 
instrament. Since the early 1970s, tests of functional abilities have been favored for population 
surveys of adultlitcracy. TTiese tests tend to have ahigh face validityforadultsbecausethey draw 
their items from everyday literacy tasks that are familiar to most English-speaking adults. The 
most recent population surveys, the Young Adultlitcracy Survey (Y ALS) (Kirsch & Jungebluc, 
1986) and the National Assessment of Literacy Sur/ey (NALS) cunendy underway, were 
constructed around short answer responses to everyday reading tasks. Although some arithmetic 
ability is required in one set of tasks and some responses require vrriting, no systematic attempt 
is made to assess these skill domains* In addition, botii skills are so embedded witiiin reading 
contexts that it is not always possible to determine where an item difficulty rests. 

In contrast, national surveys of elementary and secondary level literacy abilities are based 
upon basic reading, writing, and mafliematical skill definition; e.g., the National Assessment of 
Educational Progress (NAEP). Although the YALS included a small group of items drawn from 
\hQ NA2P reading assessment for linkage purposes, the two types of surveys differ suf ficientiy 
to make them noncomparable. That is, one can detennine how well a young adult might do on 
flieNAEPreadmg scale, based upon responses to the NAEP items in the YALS, but one cannot 
from these data equate scores on the NAEP reading assessment with scwes on the YALS 
document, quantitative, or prose scales. This incompatibility between tiie NAEP reading 
assessment and the YALS (and NALS) impedes longitudinal analyses of functional literacy 
abilities. For example, reading scores from the NAEP high school reading tests cannot be 
compared to functional literacy scores from the same individuals, obtained in the years after 
leaving high school 



In most literacy testing, 
no systemoHc effort is 
made to assess wriHng 
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There is a basic 
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SIdiis assessments 



Most tests used for pre- and post-testing of adult skill abilities in literacy i»ograms have been 
basic skills tests, most often developed from similar tests for children. A typical adult reading 
test, for example, will test various types of comprehension (e.g., literal, inferential, critical), plus 
vocabulary and perhaps stnicniral analysis (e.g., prefixes, sufBxes). These tests tend to report 
scores in grade-level equivalents, although other reporting procedures are possible. Grade-level 
equivalents arc questionable scalings for elementary and secondary level assessment; for adults 
they arc even morc problematic (Sticht, 1988; Kirsch & Gutiirie, 1977-78; Ryan & Furlong, 
1975). A sixth grader who reads at a sixth-grade level and an adult who reads at a sixth-grade 
level usually have widely difiercst reading abilities and requirc differcnt fonns of instruction, 
yet by grade-Ievd measures they arc classed as identical. In addition, grade-level designations 
foradultsarcmislcadingin that tiiey assume tiieaverageexperientialandl)ackgroundknowl^^ 
levelsofchildrcnatthedesignated grade points. Although no widelyacccptedaltemativescaling 
for adult abilities has yet been developed, ibc need for such a replacement is strong. 

If functional literacy tests werc simply alternative means fort^)ping the same skills as those 
assessed by basic skills tests, die two could be equated. However, fc r adults in ABE programs, 
\bc coirelation between the document scale of tbc Test of Adult Literacy Skills (TALS) (S imon 
& Schuster, 1990) and ti:c reading comprehension scale of tiie Tests of Adult Basic Education 
(TABE) ((TTB/McGraw-Hill, 1987) is less than .70. A regression analysis found tiipr the best 
predictorof document scale perfonnance on die TALS was tiieTABEMatiiematical Concepts 
and Applications; the TABE Reading Comprehensioi scores also made a significant but far 
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smaUer contribution to the predicted scores (Sabatmi, Venezky, & Bristow, 1992). Functional 
Uteracy tests like the TALS require not only general reading comprehension, but also problem 

^^^t^^nrfunctional Uteracy tests are not derived ftom theoretical models of skill abiUty. Functional literacy tesh 

Instead,theyaredeveloped&ommatricesoftexttypesandeveiydayUteracytasks. Forexample, are derived from matri- 

onemightdecidethatwantadsinnewspapersareanimportanttexttype(orgenre) forassessmenL ces of text Jypes and 

Many different tasks might then be defined for such texts, based upon authentic use of even <Jay literacy 

advertisements. AwantadrequestingahousecleanerandstatinganinitialsalaryofSlO.OOper not froni theoreftcal 

hourandaphonenumbermightbecoupledwithany of thefoUowingquestions: (l)\Vhatphone models of skill ability, 
number should you call to respond to the ad? (2) If you took the job and worked one day each 
week for four hours, how much would you earn in eight weeks? (3) If you worked at this job for 
a year and received positive reviews of your work fiom your employer, what salary would you 
expect for the next year if the average wage increase in your category were 6%? 

niese questions vary not only in complexity but also in the skills required. It would be The instruchonal 

misleadingtoclaimthatadultscouldorcouldnothandlewantadsonthebasisoftheiranswers implicahons ot 

to these questions. In this example, only one want ad is proposed for testing: others of greater funchonal literacy assess- 

orlessercomplexitycjuldalsobedeveloped. More importantly, sinceno analysis has been done ments are not clear, 
of the skills required to do such tasks, the instructional impUcations of such assessments are not 
obvious. Considerable research remains to be done on the interaction between text and task 

difficulty and on the skills that are involved in functional Uteracy tasks. ^ 

The incompatibiUty between basic skiUs tests andnational literacy surveys places a special The mconripatibility 

burden on poUcymakers. TTie NALS results, which wiU be avaUable within a year, will not be between basse ski Is 

compatiblewithscoresreportedbyUteracyprogramsthatusebasicskillstests,andwhileNALS tests ond national 

scotescould be converted to a crude grade-level equivalency, this would have Utde vaUdity in literacy sur/eys places 

thatmostoftheNALS tasks are nottaughtextensivelyinschool. Furthermore, the scores would a special burden on 

stillnotbeanymoiecomparablethanmathandreadingscoresthatweremappedontograde-level policymokars. 

scales. , . 

Oneoption for resolvingthis incompatibiUty is to shiflnationaladultUteracysurveys to basic 

skills tests. Another option is to develop a stronger theoretical base for functional Uteracy tests, 

one Uiat could aUow instructional interpretations of test results, and use tiiis type of test for bofli 

national and program assessments. 



Placement Tests 



Many ABE programs cunenfly use tests Uke the TABE for placement At one site, for Current placement 

example,newstudentsarefirstgiventiieTABElocator,wWchrequiresaminimumof35minutes testing ""eHjods may 

foractainistration. Basedupontiieirlocatorscores(vocabularyandarithmetic),tiiestudentstiien toke up to three hours ot 
receivefouradditionaltestsataparticulardifficultylevel. n>ese tests require an additional 2 

hours and 25 minutes of testing time. From tiie resulting scores, placements are tiien made into them mtoiust tour levels 

oneoftiireelevelsof ABEclassesorintoaGEDclass. Totalstudenttestingtimeisabout3hours; ot mstruction. 

gradmg and otiier administration time for flie staff might be 6 hours or mote for each group of 

20 students tested. . . n 

Oneproblemwifliplacementtestingisthatmanyadults whoareentermgUteracyprograms A student $ poor 

for tiie fust time have poor test-taking skiUs. They also tend to have low self-confidence and tesl-loking skills 
associatetestingwitiitiieirgeneraUynegativcfeeUngsaboutfoimalschooUng. TTiesefactotslead may leod to low 
tolowreUabiUty fortestsgivenpriortoinstruction. Testsgivenaftersevcralweeksof instruction reliability tor tests 
mightyieldmorereUablescotes,butof couisewouIdnotbcusefiilforplacemenL given pnor to 

3utgiventhesmaUnumberofplacementleveIsinvolvedinadultprograms,theneedforairee instruction, 
hours of placement testing is questionable. Futthetmote, a .tody cutientiy underway that is 
comparing placement predictions for the TABE locator, TABE reading, and TALS document 
andquantitativescales, has foundno advantage for thefuUTABEreadingbattery or eitiier of tiie 
TALS scales over tiie TABE vocabulary locator test (Venezky, Sabatini, & Bristow, in prep.), 
niat is tiie TABE vocabulary locator, which requites about 17 minutes for administration, 
predicted tiie actual placements for 125 adults in ABE and GED classes as weU as or betta- tiian 
tiieTABEreadingand vocabulary tests, eitiier separately or combine, oreitiieroftiieTALS tests. 
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Given these results and the highly negative impact of fonnal testing on adults returning to Alternatives to lengthy 
instnictional programs, alternatives to lengthy fcmnal placement testing need to be considered, formal placement 
One option is self-assessment Techniques might be develc^)ed for assisting adults in assessing testing need to be 
their own literacy abilities through sample texts and reading tasks. This might also be done considered, 
through exposure to the materials used in the first two weeks of each placement leveL A second 
optionisaself-administeredtestthatanadultmightdoathonie, atwak;oreven inalibrary, with 
or without the administrative assistance of a family member or friend. Given that the adult must 
live with the consequences of the test, motivation for cheating would probably be quite low. 



Program Evaiuafion 



One reason for administering fonnal tests at the beginning of an instructional program is to Pre-test scores tend to be 
meetreportingrequirementsforstudentprogressandprogramoutCOTies. Typically, basic skills artificially lov/; gain 
gainscoresarereporteA Therearemanyproblemswiththis,andtheresults,ascurrentlyreported, scores may then be 
have relatively limited use for policymakers. First, the prc-testscoces are based on a test that is artificially high. 
generaUy administered within the first week that an adult has enroUed in a prograi^ Formany 
adults this is the first encounter with formal testing since dropping out of high school (or grade 
school). Scorestend, therefore, to beartificially low: adultslack test-taking skills, have low self- 
confidence, and lack practice with some literacy skills. After even a few weeks in class, scores 
may rise significantly due simply to exposure to test formats, printed vocabulary, and the like. 
Reports frrai New York City ABE programs show relatively large Srsl year gams (9.9 months 
average), but less than half of this amount for a second year in a program (3.3 months average) 
and even less for those who remain for a third year (3.0 months average) (Literacy Assistance 
Center, 1991). 

Second,almostaUtestscorereportingisforreadmgccmiTOhension,yetmostABEcourses There is an incomplete 
stress a variety of skills, including basic mathematics, reading vocabulary and comprehension, match between course 
and writing, as well as specific survival skills (flying for a job, interacting witii your child*s content and test content, 
school, etc.). This means an inconq)leie match between course content and test content Pait of 
die problem witii current practices is tiiat ABE programs are attempting to serve two distina 
goals. One is to prepare adults who have not had the advantages of foraial education or did not 
make full use of tiiem to pro gress in a fonnal educational system. This patii might lead to a GED 
and, for some, to community college or a four-year institution. The second goal is to give adults 
practical skills tiiey need for self-esteem, work, home, and civic needs. In tius patii, less attention 
is given to academic skills, which means less time to abstractions, themes, and die like. (For a 
discussion of functional vs. general or academic literacy instruction, see Sticht, Amistrong, 
Hickey, & C:aylor, 1987). 

For evaluation of progress, several procedures should be considered. For academic Like narional surveys, 
programs, what is important is to know how many eventually reach tiieir academic goals (GED, program evaluations 
college, etc.) and in how much time. This infonnation would allow policymakers to detennine often ignore v/riHng and 
die overall effectiveness of tiie system as well as die relative effectiveness of different programs mathemaHcs. 
in helping students reach tiieir academic goals. The GED examination and college entry 
requirements generally serve as checks on actual skill attainment, altiiough witii open admis- 
sions, it may also be important to track progress witiiin post-secondary education. 

Students enrolled in courses tiiat stress practical skills could be assessed at tiie teraiination 
of flieir instruction with functional literacy tests. To measure actual progress, post-test scores 
might be compared to scores obtained after a montii or so of enrollmeat But we have littie 
agreement on what average gains to expect for different entry-level abilities and instructional 
regimens. And gain scores do lot translate easily into policy. A gain of 50 points on die TALS 
document scale, for example, might be a positive sign for someone vfbo entered a program witii 
low reading ability and re-cnrolled for tiie next higher level of instruction* For a person who 
entered witii die same low level of ability and planned not to proceed to the next level, tiiis m^ y 
not be a positive sign, particularly if tiie exit level of functioning were still too low to ensure 
adequate handling of print material for home, woric» and civic life. 

An alternative is to report a combination of exit scores and enrollment statistics, including 
die percentage of students who enroll in tiie next higher level course, when one is available . Hiis 
^proach requires further work, however, on matching functional literacy scores witii levels 
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of functioning in society. What policymakers need to know, for those v/ho enroll in literacy 
courses, is how many stay long enough to reach desiredlevels of functioning. At present many 
students who enroll do so on a casual basis, attending sporadically and often leaving without 
fonnally withdrawing. 

Another complication to consider is that what a functional literacy test or a basic skills test 
assesses may no t be what particular courses teach oc what particular students want to learn. For 
efficiency we would like to have assessment instruments that could infcmn instmction as well 
aspolicy. However, this may not be possible. InteUigeatpolicyrequiresdataoHnparableacross 
time and geographic region. Furthermore, policy related results mustbe interpretable in relation 
to societal goals. Students who enroll in literacy courses to learn how to read to their children 
may not care at the end of instruction what their reading levels are relative to national standards. 
However, policymakers do. One option is to divide students according to their expressed goals 
into general literacy suidents and specialized skill (or functional literacy) students. Then, only 
the former would be tested with nationally standardized instruments. The latter group might not 
beassessedatallorbeassessedthroughaltemativemeans(e.g^Lytle&Wolfe, 1989). Asecond 
option is to require that all sOidents be tested on exit with nationally standardized instruments 
as a condition of enrollment This by itself, however, will not solve the problem created by those 
who drop out before exit testing. 

Fin' Jly , we should note that so far we have discussed primarily the assessment of cogni tive 
skills. Attiuides and beliefs are also important fw literacy development, yet we have limited 
instrumentation for valid and reliable assessment in this domain. One task that remains is to 
synthesize the literature on attinides and beliefs to determine what, if anything, needs to be 
assessed and what the complexities are in doing this assessment 



What policymakers need 
to know, for those who 
enroll in literacy courses, 
is how many stay long 
enough to reach desired 
levels of funcHoning. 



We have limited instru- 
mentation for valid and 
reliable assessment of 
attitudes und beliefs. 



Score Reporting 



As stated earlier, grade-level reporting for test scores needs to be reconsidered. If it is an 
adequate scale for reporting adult abilities, then it needs to be justified both empirically and 
logically. If not, then ascale (orscales) thatappliesmoredirecdy toadultfunctioning levels needs 
to be developed. Such a scale might be developed around materials and tasks that represent 
different levels of functioning in work, home, and civic life. Alternatively, the scale might be 
built around knowledge of cognitive processing — the skills and strategies required for different 
levels of ability. AsufBcientbaseaheadyexistsforinitiatingwoiicon the latter. Studiesofwoid 
identification reaction times, automaticity in decoding, reading flexibility, and metacognition, 
in particular, provide insights into some of the cognitive skills required for reading (e.g., 
Veilutino&Scanlon, 1987;Perfetti, 1985; Stanovich, 1982). Atargetedreseaichprogramcould 
pursue these and other leads with adults, not just for reading but also for writing. 

The differences between these two jq)proaches to the development of an alternative literacy 
scale for adults should be subjected to critical examination. The materials and tasks approach 
will be easier to relate to home, work, and civic functioning; the cognitive processing approach 
wUl be easier to relate to elementary and secondary assessment and to instruction. The former 
will need to be re-evaluated as the literacy demands of everyday life change while the latter will 
require an additional step to relate it to everyday literacy demands. 



Grade-level reporting 
for test scores needs to 
be reconsidered. 



The materials and tasks 
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CONCLUSIONS 




We have attended in this synthesis to placement, achievement, program, and national survev 
needs i;^ adult literacy. Among tt^e issues we feel need to be explored in the development of more 
infomiative literacy testing are: 

• the fundamental differences between skill-based tests and functional literacy tests; 

• the varying goals and needs of different types of literacy programs; 

• the lack of empirical support for grade-level score designations for adults; 

• the limited abiUtydiscriniination required for placement into Uteracycour^ 

• the difficulties in interpreting gain scores; and 

• the limited attention currently given to testing of writing and mathematic. 



What we have notdiscussed in depth is testingthebasichypothesisbehindUteracyfundm — 
that program impacts are sufficient to justify the types and amounts of funding currently 
allocated This is a complicated question that can only be answered after the issues raised here 
areresolvedWithoutanaccuratemeasure of individual pn)gressthatlinks to national trendda^ 
and without linking both to functional levels, little can be decided about long-term impacts of 
programs. Even with these measures, other factors would need to be considered. The mix of 
available jobs, how well employers utilize workers with improved skill levels, social mobility, 
and all of the factors that impinge upon family stability would need to be considered. 

Foe example, adults may become more skilled through particular literacy iwograms, but 
because of the strength of the dollar abroad or other factors, jobs may iK>t be available foe many 
of the students wanting to enter the labor market Therefwe, by some evaluation measures these 
literacy programs would be deemed unsuccessful. Altematiyely, workers may, through 
workplace programs, acquire new abilities but rigid wOTk organization may not allow these new 
abilities to be utilized and consequently program impacts on salary, performance, and safety 
might be attenuated. Similar arguments can be made about long-term inq)acts upon home and 
civic functioning. Literacy is not the only factor that determines how well a person functions in 
these areas. Multiple indicators will probably be needed for all impact evaluations, with carefiil 
modeling of the influence of uncontrolled factors. 



Without an accurate 
•neasure of individual 
progress that links to 
national trend data, and 
without linking both to 
functional levels, little can 
be decided about long-term 
impacts of programs* 



Literacy is not the only factor 
that determines how well a 
worker functions* 
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